1. Field of the Invention
This invention relates generally to multiframe image reconstruction techniques and, more particularly, to the adaptive acquisition and/or display of image frames using multi-focal displays.
2. Description of the Related Art
Real world scenes contain an extremely wide range of focal depths, radiance and color and thus it is difficult to design a camera capable of imaging a wide range of scenes with high quality. To increase the versatility of its imaging system, most cameras have adjustable optical settings, such as the focus, exposure, and aperture. In most such systems, the camera includes some form of automatic adjustment of these settings depending on the object scene, such as auto-focus (AF), automatic gain (AG), and auto-exposure (AE) algorithms. These automatic algorithms typically use image data to perform adjustment. The camera will capture multiple images under different acquisition settings until it finds the optimal settings for a single image. The adjustment process often consumes significant power to adjust the focus and aperture settings. Finding efficient algorithms for automatically adjusting the camera settings is thus important for minimizing power consumption as well as improving performance for the user.
Traditional settings adjustment algorithms rely on multiple tests in order to find the best settings for acquiring a single image. A large class of alternate image processing algorithms, known as multiframe reconstruction algorithms combine a set of multiple images to synthesize a single image of higher quality. Such multiframe algorithms operate on a set of images where each image contains different information about the scene. The reconstruction algorithm combines these multiple sources of information, typically based on information about the source of the image variations (shifts, defocus, exposure level, etc.) to form a single reconstructed image. Typically, the set of images is captured using predetermined acquisition settings. In other words, the acquisition settings do not depend on image content. The traditional problem addressed by multiframe reconstruction is then, given the set of already acquired images, synthesize the best quality reconstructed image from the set of available images.
The choice of acquired images, however, can significantly affect the quality of the final reconstructed image. Multiframe reconstruction combines different information from different images into the single reconstructed image. However, if no image in the set has collected certain information, then that information cannot be represented in the reconstructed image. More generally, some visual information is more important than other information when constructing an image of a particular scene.
Multi-focal displays (MFDs) are one device that can implement multiframe reconstruction. MFDs typically use rapid temporal and focal modulation of a series of 2-dimensional images to render 3-dimensional (3D) scenes that occupy a certain 3D volume. This series of images is typically focused at parallel planes positioned at different, discrete distances from the viewer. The number of focal planes directly affects the viewers' eye accommodation and 3D perception quality of a displayed scene. If a given 3D scene is continuous in depth, too few planes may make the MFD rendering look piecewise with discontinuities between planes or result in contrast loss. More planes is typically better in terms of perceptual quality, but can be more expensive to implement and often may not be achievable because of practical display limitations including bandwidth and focal modulation speed.
Therefore, an important consideration for MFDs is the focal plane configuration, including the number of focal planes and the location of the focal planes (that is, distances from the viewer). Multi-focal displays typically use focal plane configurations where the number and location of focal planes are fixed. Often, the focal planes are uniformly spaced. This one size fits all approach does not take into account differences in the scenes to be displayed and the result can be a loss of spatial resolution and perceptual accuracy.
Therefore, there is a need for multiframe reconstruction techniques that actively select which images should be acquired, in addition to combining the acquired images into a reconstructed image. There is a need for better approaches to determining focal plane configurations for multi-focal displays.